CFP 1457 US 
CRF-312/313 
BIF022322 US 



PATENT 



APPLICANT 



Felix HENRY 



TITLE 



Digital signal coding with division into tiles 



Priorities claimed 

French patent applications No 9916016 and 9916021 
of December 1 7, 1999 



1 



5 



10 The present invention in general terms concerns digital signal 

coding. 

The purpose of coding is to compress the signal, which makes it 
possible to transmit, or respectively store, the digital signal whilst reducing the 
transmission time or transmission rate, or respectively reducing the memory 
15 space used. 

The invention is situated in the field of compression of digital 
signals with loss. 

In the particular case of a fixed image signal, the coding technique 
normally referred to as JPEG (Joint Photographic Expert Group) is very simple 
20 to use. This is because, this technique having been designed for coding 
"natural" images in colours, it proposes a single parameter to be set by the 
user, namely the ratio of compression to quality. 

New compression techniques have now been developed, such as 
the one referred to as JPEG2000, in order to meet specific requirements 
25 existing within a vast range of applications, such as medical imaging, 
synthesised images, digital photography or satellite imaging, for example. In 
order to deal with very different types of image, a large number of adjustment 



parameters are available, in order to adapt the process to each particular type 
of application. 

Notably, some of these parameters are parameters for partitioning 
the image into tiles, or sub-images. The use of sub-images makes it possible 
to reduce the memory space occupied by the data currently being processed 
during the coding and decoding of an image. In addition, the sub-images allow 
random access in the file containing the compressed data. 

However, the large number of parameters to be set makes the 
use of such a compression technique complex. In addition, the user chooses 
the partitioning into tiles without knowing where the tile limits will be situated 
compared with the semantic content of the image. 

The document US 5 815 168 offers the possibility of modifying the 
shape of tiles in an image according to the display mode or other display 
factors. 

The present invention aims to remedy the drawbacks of the prior 
art by providing a method and device for dividing a digital signal, according to 
which the setting of the parameters for division into tiles is simplified. 

In general terms, the invention concerns a method of dividing a 
digital signal representing physical quantities, characterised in that it includes 
the steps of: 

- displaying a representation of the signal, 

- acquiring at least one parameter for a partitioning of the signal, 

- displaying the representation of the signal and of the partitioning 
of the signal corresponding to said at least one previously acquired parameter. 

To this end, the invention proposes a method of dividing a digital 
signal representing physical quantities, characterised in that it includes the 
steps of: 

- determining an initial partitioning of the signal, 



- displaying a representation of the signal and the previously 
determined signal partitioning, 

- acquiring at least one partitioning modification parameter, 

- modifying the partitioning of the signal. 

By virtue of the invention, the setting of the parameters for division 
into tiles is very simple, since the signal and the partitioning of the latter into 
tiles are displayed, which enables a user to display the partitioning and the 
modification thereof. 

According to a preferred characteristic, the partitioning of the 
signal includes blocks of samples of the signal, and said at least one 
modification parameter is chosen from amongst a block height and a block 
width. 

This implementation is simple and fast. 

According to another preferred characteristic, which can be 
combined with the previous one, said at least one modification parameter 
makes it possible to translate the partitioning with respect to the signal. 

Thus the size of the tiles, or blocks, is not modified, only their 
limits are moved. 

According to a preferred characteristic, the modified partitioning of 
the signal is selected from a predetermined set of partitionings. 

According to a preferred characteristic, the invention also includes 
the step of simulating the coding an decoding of the signal and the displayed 
representation of the signal is the result of the step of simulating. 

According to a preferred characteristic, distortions in the 
representation of the signal are emphasized. 

The user can see the distortions due to the coding and decoding 
of the signal, those distortions being possibly emphasized. Thus, the user can 
take into account the distortions in order to modify the setting of the parameters 
for division into tiles. 
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The invention also concerns a method of coding a digital signal 
representing physical quantities, which includes the division method presented 
above. 

The coding method has advantages similar to those previously 

5 disclosed. 

Correlatively, the invention proposes a device for dividing a digital 
signal representing physical quantities, characterised in that it has: 

- means of determining an initial partitioning of the signal, 

- means of displaying a representation of the signal and of the 
10 previously determined signal partitioning, 

- means of acquiring at least one partitioning modification 

parameter, 

- means of modifying the partitioning of the signal. 

15 In another aspect, the invention aims to provide a method and a 

device for dividing a digital signal, according to which the setting of the 
parameters for division into tiles is adapted to the semantic content of the 
signal. 

To this end, the invention proposes a method of dividing a digital 
20 signal representing physical quantities, characterised in that it includes the 
steps of: 

- determining at least one area of interest in the signal, 

- determining an initial partitioning of the signal, including 
partitioning areas, 

25 - modifying the partition of the signal according to said at least one 

area of interest and a predetermined criterion. 
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Thus the setting of the parameters for dividing into tiles is adapted 
to the semantic content of the signal, since the area or areas of interest are 
taken into account during the formation of the partitioning of the signal. 

According to a preferred characteristic, the partitioning of the 
5 signal is modified so that said at least one area of interest is not divided into 
two partitioning areas. 

Thus an area of interest is situated entirely within a partitioning 
area, and consequently will be processed on a single occasion, for example 
during the coding and decoding of the signal. In addition, any discontinuities 
10 which may exist at the boundaries of the partitioning areas do not affect the 
areas of interest. 

According to another preferred characteristic, the partitioning of 
the signal is modified so that the partitioning areas are the smallest possible in 
order to satisfy the predetermined criteria. 

15 Thus the number of partitioning areas is as large as possible, 

having regard to taking the areas of interest into account. The memory size 
necessary for processing each partitioning area is thus reduced. 

According to a preferred characteristic, the partitioning of the 
signal includes blocks of samples of the signal, and the modification of the 
20 partitioning includes the modification of at least one parameter chosen from 
amongst a block height and block width. 

This implementation of the invention is simple and fast. 
According to another preferred characteristic, which can be 
combined with the previous one, the modification of the partitioning includes a 
25 translation of the partitioning with respect to the signal. 

Thus the size of the partitioning areas is not modified, their limits 
are merely moved so that the areas of interest are not shared between two 
partitioning areas. 
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According to a preferred characteristic, the modification of the 
partitioning results in a modified partitioning which is selected from a 
predetermined set of partitionings. 

The invention also concerns a method of coding a digital signal 
5 representing physical quantities, characterised in that it includes the previously 
disclosed division method. 

The coding method has advantages similar to those previously 

disclosed. 

The invention also concerns a device for dividing a digital signal 
1 0 representing physical quantities, characterised in that it has: 

- means of determining at least one area of interest in the signal, 

- means of determining an initial partitioning of the signal, 
including partitioning areas, 

- means of modifying the partitioning of the signal according to 
1 5 said at least one area of interest and a predetermined criterion. 

The invention also relates to a coding device which includes the 
division device previously disclosed. 

These devices have means adapted to implement the above 
characteristics. 

20 The invention also concerns a digital apparatus including the 

division or coding device, or means of implementing the division or coding 
method. The advantages of the device and of the digital apparatus are 
identical to those previously disclosed. 

The invention also concerns an information storage means, which 
25 can be read by a computer or a microprocessor, integrated or not into the 
device, possibly removable, storing a program implementing the division or 
coding method. 
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The characteristics and advantages of the present invention will 
emerge more clearly from a reading of a preferred embodiment illustrated by 
the accompanying drawings, in which: 

- Figure 1 is an embodiment of a device implementing the 

5 invention, 

- Figure 2 depicts a coding device according to a first embodiment 
of the invention and a corresponding decoding device, 

- Figure 3 is a first embodiment of the coding method according to 

the invention, 

10 - Figure 4 is a representation of an image and a partitioning 

thereof, 

- Figure 5 is a representation of an image and a partitioning 

thereof, 

- Figure 6 depicts a coding device according to a second 
15 embodiment of the invention and a corresponding decoding device, 

- Figure 7 is a second embodiment of the coding method 
according to the invention, 

- Figure 8 is a representation of an image processed according to 
the second embodiment, 

20 - Figures 9a and 9b are representations of an image processed 

according to a third embodiment, 

- Figure 10 depicts a coding device according to a fourth 
embodiment of the invention and a corresponding decoding device, 

- Figure 11 is a fourth embodiment of the coding method 
25 according to the invention, 

- Figure 12 is an embodiment of a method of determining the 
partitioning of the image, included in the coding method of Figure 11, 
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- Figure 13 is a representation of an image and an area of interest 

therein, 

- Figure 14 is a representation of an image and a partitioning 

thereof, 

5 - Figure 15 is a representation of an image and a partitioning 

thereof. 

According to the chosen embodiment depicted in Figure 1, a 
device implementing the invention is for example a microcomputer 10 
10 connected to different peripherals, for example a digital camera 107 (or a 
scanner, or any means of acquiring or storing an image) connected to a 
graphics card and supplying information to be processed according to the 
invention. 

The device 10 has a communication interface 112 connected to a 
15 network 113 able to transmit digital data to be processed or conversely to 
transmit data processed by the device. The device 10 also has a storage 
means 108 such as for example a hard disk. It also has a drive 109 for a disk 
110. This disk 1 10 can be a diskette, a CD-ROM or a DVD-ROM, for example. 
The disk 110, like the disk 108, can contain data processed according to the 
20 invention as well as the program or programs implementing the invention which, 
once read by the device 10, will be stored on the hard disk 108. According to a 
variant, the program enabling the device to implement the invention can be 
stored in the read only memory 102 (referred to as ROM in the drawing). In a 
second variant, the program can be received in order to be stored in a manner 
25 identical to that described previously by means of the communication network 
113. 

The device 10 is connected to a microphone 111. The data to be 
processed according to the invention will in this case be from the audio signal. 

This same device has a screen 104 for displaying the data to be 
30 processed or serving as an interface with the user, who can thus parameterise 
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certain processing modes, using the keyboard 114 or any other means (a 
mouse, for example). 

The central unit 100 (referred to as CPU in the drawing) executes 
the instructions relating to the implementation of the invention, instructions 
5 stored in the read only memory 102 or in the other storage elements. During 
powering up, the processing programs stored in a non-volatile memory, for 
example the ROM 102, are transferred into the random access memory RAM 
103, which will then contain the executable code of the invention as well as the 
registers for storing the variables necessary for implementing the invention. 

10 In more general terms, an information storage means, which can 

be read by a computer or by a microprocessor, integrated or not into the device, 
possibly removable, stores a program implementing the method according to 
the invention. 

The communication bus 101 affords communication between the 
15 different elements included in the microcomputer 10 or connected to it. The 
representation of the bus 101 is not limitative and notably the central unit 100 is 
capable of communicating instructions to any element of the microcomputer 10 
directly or by means of another element of the microcomputer 10. 

Several embodiments of the invention will be detailed in the 

20 following. 

With reference to Figure 2, a first embodiment of a coding device 
3 according to the invention is intended to code a digital signal for the purpose 
of compressing it. The coding device is integrated into an apparatus, which is 
for example a digital photographic apparatus, or a digital camcorder, or a 
25 database management system, or a computer. 

The device according to the invention has a signal source 30, here 
an image signal IM, whether a fixed image or an image sequence. In general 
terms, the signal source either contains the digital signal, and has for example a 
memory, a hard disk or a CD-ROM, or converts an analogue signal into a digital 
30 signal, and is for example an analogue camcorder associated with an analogue 
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to digital converter. The image source 30 generates a series of digital samples 
representing an image IM. The image signal IM is a series of digital words, for 
example bytes. Each byte value represents a pixel of the image IM, here with 
256 grey levels or in colour. 

An output of the signal source 30 is connected to a circuit 31 for 
selecting parameters for partitioning the image into tiles, or blocks. A tile is a 
set of samples of the image, for example forming a rectangle. The tiles are 
adjacent. The present invention concerns particularly the selection of 
parameters for partitioning the image into tiles. 

The circuit 31 is connected to an image and partitioning display 
circuit 310, which enables a user to display the result of the partitioning. 

The circuit 31 makes it possible to determine an initial partitioning 
of the signal, which will be displayed by the circuit 310, just like the image. 

The circuit 31 next makes it possible to acquire at least one 
partitioning modification parameter, and to modify the partitioning of the signal 
accordingly. 

The circuit 31 is connected to a circuit 32 for selecting other 
parameters. These parameters include notably a required compression ratio, a 
type of wavelet decomposition, a number of decomposition levels, and a coding 
with or without loss. These parameters are selected by the user via an 
interface, or are read in a memory in which they were previously stored. 

ft should be noted that, in an equivalent manner, the circuit 32 can 
be arranged in front of the circuit 31 . 

The circuit 32 is connected to a coding circuit 33, which effects a 
coding, known per se, of the partitioned image. For example, the coding used 
is in accordance with the standard JPEG2000 (Joint Photographic Expert 
Group), currently being produced, in which the digital signal can be 
decomposed into tiles, each tile being a sub-image. The coding includes a 
quantisation and an entropic coding such as an arithmetic coding or a Huffman 
coding. 
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The coding circuit 33 is connected to a coded data processing 
circuit 34, which stores and/or transmits the compressed file containing the 
coded image to a decoding device 4. 

The decoding device 4 has a coded data reception circuit 41 . The 
circuit 41 is connected to a decoding circuit 42, which performs operations 
which are the reverse of those of the coding circuit 33. The decoding circuit 42 
is connected to a circuit 43 for using the decoded data, for example in order to 
display a decoded image. 

Figure 3 depicts a first embodiment of a method of dividing and 
coding an image, according to the invention. This method is implemented in 
the coding device (Figure 2) and includes steps E1 to E9. 

The method is implemented in the form of an algorithm which can 
be stored in whole or in part in any information storage means capable of 
cooperating with the microprocessor. This storage means can be read by a 
computer or by a microprocessor. This storage means is integrated or not into 
the device, and may be removable. For example, it may have a magnetic tape, 
a diskette or a CD-ROM (fixed-memory compact disc). 

In general terms, the method includes the steps of: 

- displaying a representation of the image, 

- acquiring at least one parameter of a partitioning of the image, 

- displaying the representation of the image and the partitioning of 
the image corresponding to said at least one previously acquired parameter. 

Step E1 is the initial partitioning of an image to be coded. The 
image is divided into rectangular tiles, with the same predetermined size. The 
width L and height H of a tile are here integer powers of two. It should be noted 
that other constraints on the size or shape of the tiles are possible in the 
context of the invention. 

The initial values of the parameters L and H are predetermined, or 
in a variant are calculated with respect to the image, for example with respect to 
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the size of the image. Thus an image can be divided systematically into four 
tiles. 

As depicted in Figure 4, the tiles are positioned on the image in 
accordance with a regular structure. The tiles are positioned as from the top 
5 left-hand corner of the image, which then corresponds to the top left-hand 
corner of a tile. 

When the size of the image does not exactly correspond to a 
whole number of tiles, the tiles on the right-hand and bottom edges of the 
image are truncated. Such a partitioning of the image is described by the two 
10 parameters L and H, which are respectively the width and height of a tile. In an 
equivalent fashion, the partitioning can be described by two other parameters 
which represent the parameters L and H. Thus, since the parameters L and H 
are integer powers of two, the partitioning can be represented by log 2 (L) and 
log 2 (H). 

15 Naturally, the partitioning can be effected with tiles of different 

shapes, or different sizes. For example, it can result from a quadtree 
decomposition. 

The following step E2 is the extraction of the representation of the 
partitioning effected at the previous step. In the example chosen, the 
20 parameters L and H are extracted from a memory in which they are stored. 

The following step E3 is the display of a representation of the 
image. This representation is preferably the image itself, or can be a modified 
version of the image, for example a sub-image extracted from the original 
image, or a reduced version of the original image, or an enlarged version of the 
25 image. 

The following step E4 is the display of a representation of the 
partitioning, in correspondence with the displayed image. As depicted in Figure 
4, the limits between tiles are depicted by lines, here dotted lines. Naturally, if a 
reduced version of the image is displayed, then the representation of the 
30 partitioning is reduced in an identical manner. 
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The following step E5 is a visual check on the match of the 
partitioning with the content of the image. This is followed by step E6, which is 
a test for determining whether the current partitioning is satisfactory, according 
to the semantic content of the image. 

If the response is negative, then this step is followed by step E7, 
which enables the user to modify the partitioning. Different means can be used 
so that the user indicates the modifications which he desires. For example, as 
depicted in Figure 5, symbols SB1 and SB2, such as arrows, are 
superimposed on the partitioning. These symbols can be repositioned by the 
user, by means of the mouse or commands entered by means of the keyboard. 
The effect of the repositioning of the symbols SB1 and SB2 is to reposition the 
tile limits. The tile sizes can be chosen from a predetermined set of sizes. 
Thus, for example, the tile limits are positioned at predefined positions such 
that the height and width of the tiles remain integer powers of two. 

Modification of the partitioning can also be a translation of the 
partitioning with respect to the image. Naturally it is possible to effect these two 
modifications in a combined fashion, that is to say to modify the tile size and to 
translate the partitioning. 

Step E7 is followed by the previously described step E4. 

When the response is positive at step E6, then this step is 
followed by the step E8 of coding the image using the current partitioning. The 
coding includes a quantisation and entropic coding of the samples of the image. 
The coded data of the image are stored in a compressed file, in which the 
parameters L and H for partitioning the image are also stored at step E9. 

Figure 6 depicts a second embodiment of the invention. The 
device of figure 6 is similar to the device of Figure 2 and includes the same 
circuits. It also includes an additional circuit 320 for simulating the coding and 
decoding of the image. The circuit 320 is connected between the circuits 31 
and 310. 



14 



This embodiment allows to display the expected distortions along 
the edges of the image. As a matter of fact, instead of displaying the original 
image and the location of the frontiers of the blocks in the partition, the device 
displays the expected decoded image. 
5 Thus , the user can visually control that the distortions along the 

edges do not interfere with the important image content. 

For this purpose, the simulating circuit 320 simulates the encoding 
and decoding of the image, in order to obtain the decoded image. 

To perform this simulation, it is possible to completely encode and 
10 then completely decode the image. As a variant it is possible to simulate 
encoding/decoding by performing all stages of image compression until 
quantization, and then dequantizing the signal and performing reverse stages. 

Figure 7 depicts a second embodiment of a method according to 
15 the invention. This method is implemented in the coding device (Figure 6). 

This embodiment includes the previously described steps E1 to 
E9. It also includes steps E41 and E42 between steps E4 and E5. 

Step E41 is the simulation of the encoding and decoding of the 

image. 

20 Step E42 is the display of the distortion to be expected in the 

image for a given image partition. 

Figure 8 represents the image together with the distortions along 
the edges of the image partition. This is what the user sees at step E42 in the 
25 second embodiment. 

Figures 9a and 9b depict a third embodiment of the invention. 
According to this embodiment, the possibility is given to the user to emphasize 
the distortions due to the partition of the image. This emphasis allows the user 
30 to better identify the distortions and their interaction with the image content. 

Figure 9a represents the image with its distortions, and Figure 9b 
represents the same image where the distortions have been artificially 
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emphasized. Distortions emphasis can be achieved by a number of methods. 
For instance, a high pass filtering can be applied along the edges of the image. 
Typical examples of high-pass filtering are given in « Fundamentals of Digital 
Image Processing », A.K. JAIN, Prentice-Hall International Editions, 1989. 
5 Another possibility to emphasize the distortion along the edges would be to 
equalize the signal along the said edges. Equalization techniques are also 
described in the same reference. 

With reference to Figure 10, a fourth embodiment of a coding 
10 device 3 according to the invention is similar to the device described with 
reference to Figure 2. Only circuits 31 and 31 1 are specific to this embodiment. 
The other circuits will not be described. 

The signal source 30 is connected to the circuit 31 which defines 
at least one area of interest in the image. The circuit 31 is connected to an 

15 image display circuit 310. As depicted in Figure 13, a representation of an 
area of interest is superimposed on the image display. The representation of 
the area of interest includes for example symbols which can be moved by a 
user by means of a mouse, for example, so as to position the areas of interest 
and to define their respective size. In the example in Figure 15, a single 

20 rectangular area of interest is depicted, but the number and shape of the areas 
may be different. 

The circuit 31 is connected to a circuit 311 for calculating the 
parameters for partitioning into tiles. A tile is a set of samples in the image, for 
example forming a rectangle. The tiles are adjacent. As detailed below, the 
25 partitioning into tiles depends on the areas of interest previously defined. 

The circuit 31 1 is connected to the circuit 32 for selecting other 
parameters. It should be noted that, in an equivalent manner, the circuit 32 can 
be arranged in front of the circuit 31 1 . 

Figure 11 depicts a fourth embodiment of a method of dividing 
30 and coding an image, according to the invention. This method is implemented 
in the coding device (Figure 10) and includes steps S1 to S3. 
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The method is implemented in the form of an algorithm which can 
be stored in whole or in part in any information storage means capable of 
cooperating with the microprocessor. This storage means can be read by a 
computer or by a microprocessor. This storage means is integrated or not into 
5 the device, and may be removable. For example, it may have a magnetic tape, 
a diskette or a CD-ROM (fixed-memory compact disc). 

Step S1 is the definition of the areas of interest. For example, at 
this step, the user is offered the possibility of defining the position, shape and 
size of each of the areas of interest which he chooses in the image. As 
10 depicted in Figure 13, the user defines at least one area of interest by means 
of two symbols, here arrows, which he moves on the image in order to 
designate two opposite corners of a rectangle, for example by means of the 
mouse. When the rectangle does indeed correspond to the area of interest, the 
user validates his choice, and step S1 is followed by step S2. 

1 5 Step S2 is the use of the areas of interest data previously defined 

by the user in order to determine tiles in the image. This step will be detailed 
below. The image is divided into rectangular tiles, with the same predetermined 
size. The width L and height H of a tile are here integer powers of two. It 
should be noted that other constraints on the size of the tiles are possible in the 

20 context of the invention. 

As depicted in Figure 14, the tiles are positioned on the image in 
accordance with a regular structure. The tiles are positioned as from the top 
left-hand corner of the image, which then corresponds to the top left-hand 
corner of a tile. 

25 When the size of the image does not exactly correspond to a 

whole number of tiles, the tiles on the right and bottom edges of the image are 
truncated. Such a partitioning of the image is described by the two parameters 
L and H, which are respectively the width and height of a tile. In an equivalent 
fashion, the partitioning can be described by two other parameters which 

30 represent the parameters L and H. Thus, since the parameters L and H are 
powers of two, the partitioning can be represented by log 2 (L) and log 2 (H). 
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Naturally, the partitioning can be effected with tiles of different 
shapes, or different sizes. For example, it can result from a quadtree 
decomposition. 

Step S2 is followed by the step S3 of coding the image using the 
previously defined partitioning. The coding includes a quantisation and entropic 
coding of the samples of the image. The image coding data are stored in the 
compressed file, in which the image partitioning parameters are also stored. 

Figure 12 depicts an embodiment of step S2 of forming the 
partitioning of the image as a function of the area of interest data. Step S2 
includes sub-steps S20 to S27. 

Step S20 is the determination of a width L and height H of a tile. 
For this purpose, the greatest area of interest width is considered, and the width 
L chosen is the width which is immediately greater than it. 

Likewise, the greatest area of interest height is considered, and 
the height H chosen is the height which is immediately greater than it. 

The following step S21 is a test for determining whether there 
exists an area of interest through which a tile limit passes. This is because the 
tiles are formed and positioned as from the top left-hand corner of the image, 
whilst the areas of interest are positioned freely on the image. Consequently it 
is possible that an area of interest may have a tile limit passing through it, and 
consequently shared between two tiles. 

If the response is positive, then step S21 is followed by step S22, 
at which the tile width is increased by a predetermined value. Step S22 is 
followed by step S23, which is identical to step S21 . 

If the response is positive at step S23, then step S23 is followed 
by step S24, at which the tile height is increased by a predetermined value and 
the tile width is returned to its previous value. Step S24 is followed by step 
S25, which is identical to step S21. 
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If the response is positive at step S25, then step S25 is followed 
by step S26, at which the tile width is once again increased by the 
predetermined value. Step S26 is followed by step S21 . 

Thus the width and height of tiles are progressively increased, 
5 until no area of interest has a tile limit passing through it. In addition, this 
method makes it possible to form the smallest possible tiles while taking 
account of areas of interest, which makes it possible to keep the largest 
possible number of tiles in the image. 

As a variant, the partitioning can also be translated with respect to 

10 the image. 

When the response is negative at one of steps S21, S23 and S25, 
then the step in question is followed by the previously described step S3, which 
effects the coding of the image with the partitioning obtained. Such a 
partitioning is depicted in Figure 15. The area of interest is completely 
1 5 included in one of the tiles of the partitioning. 

Naturally, the present invention is in no way limited to the 
embodiments described and depicted, but quite the contrary encompasses any 
variant within the capability of a person skilled in the art. 



20 



